Skip to content

[FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] Adapt HPU scheduler, ngram proposer and offloading connector tests to upstream API drift - #1556

Merged
iboiko-habana merged 4 commits into
vllm-project:mainfrom
pawel-olejniczak:fix/batch-cycle-202606
Jun 22, 2026
Merged

Conversation

@pawel-olejniczak

@pawel-olejniczak pawel-olejniczak commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Bug 1: Forward throttle_prefills in HPUAsyncScheduler.schedule

  • State machine id: hpu_async_scheduler_schedule_positional_arg
  • Commit: 957ba4d

Root cause

vLLM PR #44558 added a throttle_prefills positional arg to Scheduler.schedule(); EngineCore calls it positionally but the HPU override only accepted self.

Upstream PR

vllm-project/vllm#44558

Fix

Accept throttle_prefills (default False) on the HPUAsyncScheduler.schedule override and forward it to super().schedule().

Bug 2: Pass num_speculative_tokens to NgramProposer.propose

  • State machine id: ngram_proposer_propose_missing_positional_arg
  • Commit: 82155ea

Root cause

vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens positional arg to NgramProposer.propose().

Upstream PR

vllm-project/vllm#32374

Fix

Prepend self.speculative_config.num_speculative_tokens in propose_ngram_draft_token_ids to match the new upstream signature.

Bug 3: Align OffloadingConnector stats tests with upstream flat-metrics API

  • State machine id: offloading_connector_cpu_to_gpu_metrics_missing
  • Commit: c1eb9e3

Root cause

vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing {types, data} flat-metric payload, dropping the per-direction CPU_to_GPU/GPU_to_CPU list shape the tests still asserted.

Upstream PR

vllm-project/vllm#35669

Fix

Rewrite test_metrics.py to exercise increase_counter/observe_histogram/aggregate/reduce/reset against the new self-describing stats contract.

Bug 4: Align OffloadingConnector scheduler flush assertions with upstream defer-on-finish

  • State machine id: offloading_connector_flush_on_finish_deferred
  • Commit: 575a178

Root cause

vLLM commit f428718ffe (PR #45823, "Defer on_request_finished until in-flight
transfers drain") changed OffloadingConnectorScheduler: a finishing request with
in-flight store jobs no longer flushes those stores immediately — finalization
is deferred until transfers drain, and flush now fires only on preemption or
block reuse. test_concurrent_lookups_of_the_same_prefix and
test_abort_loading_requests still asserted flush-on-finish, so they failed once
the target vLLM SHA picked up #45823.

Upstream PR

vllm-project/vllm#45823

Fix

Drop the stale expected_flushed_gpu_block_indexes assertions in the two affected
tests (matching upstream's own equivalents, which assert no flush in these
scenarios). test_request_preemption keeps its flush-on-preemption assertion,
which upstream still honors.

…d throttle_prefills in HPUAsyncScheduler.schedule

Root cause: vLLM PR #44558 added a throttle_prefills positional arg to Scheduler.schedule(); EngineCore calls it positionally but the HPU override took only self.
Upstream: vllm-project/vllm#44558
Fix: accept throttle_prefills (default False) and forward it to super().schedule().

Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
…um_speculative_tokens to NgramProposer.propose

Root cause: vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens positional arg to NgramProposer.propose().
Upstream: vllm-project/vllm#32374
Fix: Prepend self.speculative_config.num_speculative_tokens in propose_ngram_draft_token_ids.

Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
…OffloadingConnector stats tests with upstream flat-metrics API

Root cause: vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing {types, data} payload with flat metric names, dropping the per-direction CPU_to_GPU/GPU_to_CPU list shape the tests still asserted.
Upstream: vllm-project/vllm#35669
Fix: Rewrite test_metrics.py to exercise increase_counter/observe_histogram/aggregate/reduce/reset against the new self-describing stats contract.

Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
@pawel-olejniczak pawel-olejniczak changed the title [FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] 3 hourly-CI fixes [FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] Adapt HPU scheduler, ngram proposer and offloading tests to upstream API drift Jun 19, 2026
…on-finish

Root cause: upstream vLLM commit f428718ffe (#45823, "Defer
on_request_finished until in-flight transfers drain") changed the
OffloadingConnectorScheduler so a finishing request with in-flight store
jobs no longer flushes those stores immediately; finalization is deferred
until the transfers drain, and flush now fires only on preemption or block
reuse.

test_concurrent_lookups_of_the_same_prefix and test_abort_loading_requests
still asserted flush-on-finish, so they failed once the target vLLM SHA
picked up #45823. Upstream's own equivalents assert no flush in these
scenarios. Drop the stale expected_flushed_gpu_block_indexes assertions;
test_request_preemption keeps its flush-on-preemption assertion, which
upstream still honors.

Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ CI Passed

All checks passed successfully against the following vllm commit:
ecf9d83520eb217401b47d8a5451a27c5231b8c2

@pawel-olejniczak pawel-olejniczak changed the title [FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] Adapt HPU scheduler, ngram proposer and offloading tests to upstream API drift [FIX_FOR_VLLM_CUSTOM=ecf9d83520eb217401b47d8a5451a27c5231b8c2] Adapt HPU scheduler, ngram proposer and offloading connector tests to upstream API drift Jun 22, 2026
@iboiko-habana
iboiko-habana merged commit c5c7a55 into vllm-project:main Jun 22, 2026
3 checks passed
slokesha pushed a commit to slokesha/vllm-gaudi that referenced this pull request Jul 6, 2026
…HPU scheduler, ngram proposer and offloading connector tests to upstream API drift (vllm-project#1556)

## Bug 1: Forward throttle_prefills in HPUAsyncScheduler.schedule

- **State machine id**: hpu_async_scheduler_schedule_positional_arg
- **Commit**: 957ba4d

### Root cause
vLLM PR #44558 added a throttle_prefills positional arg to
Scheduler.schedule(); EngineCore calls it positionally but the HPU
override only accepted self.

### Upstream PR
vllm-project/vllm#44558

### Fix
Accept throttle_prefills (default False) on the
HPUAsyncScheduler.schedule override and forward it to
super().schedule().

## Bug 2: Pass num_speculative_tokens to NgramProposer.propose

- **State machine id**: ngram_proposer_propose_missing_positional_arg
- **Commit**: 82155ea

### Root cause
vLLM PR #32374 (Dynamic SD) added a leading num_speculative_tokens
positional arg to NgramProposer.propose().

### Upstream PR
vllm-project/vllm#32374

### Fix
Prepend self.speculative_config.num_speculative_tokens in
propose_ngram_draft_token_ids to match the new upstream signature.

## Bug 3: Align OffloadingConnector stats tests with upstream
flat-metrics API

- **State machine id**: offloading_connector_cpu_to_gpu_metrics_missing
- **Commit**: c1eb9e3

### Root cause
vLLM PR #35669 rewrote OffloadingConnectorStats to a self-describing
{types, data} flat-metric payload, dropping the per-direction
CPU_to_GPU/GPU_to_CPU list shape the tests still asserted.

### Upstream PR
vllm-project/vllm#35669

### Fix
Rewrite test_metrics.py to exercise
increase_counter/observe_histogram/aggregate/reduce/reset against the
new self-describing stats contract.

## Bug 4: Align OffloadingConnector scheduler flush assertions with
upstream defer-on-finish

- **State machine id**: offloading_connector_flush_on_finish_deferred
- **Commit**: 575a178

### Root cause
vLLM commit f428718ffe (PR #45823, "Defer on_request_finished until
in-flight
transfers drain") changed OffloadingConnectorScheduler: a finishing
request with
in-flight store jobs no longer flushes those stores immediately —
finalization
is deferred until transfers drain, and flush now fires only on
preemption or
block reuse. test_concurrent_lookups_of_the_same_prefix and
test_abort_loading_requests still asserted flush-on-finish, so they
failed once
the target vLLM SHA picked up #45823.

### Upstream PR
vllm-project/vllm#45823

### Fix
Drop the stale expected_flushed_gpu_block_indexes assertions in the two
affected
tests (matching upstream's own equivalents, which assert no flush in
these
scenarios). test_request_preemption keeps its flush-on-preemption
assertion,
which upstream still honors.

---------

Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
Signed-off-by: slokesha <slokeshappa@habana.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants